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ABSTRACT 

Highly  efficient  command  post  teams  are  essential  to  efficient  execution  of  network 
centric  warfare  operations.  Unfortunately,  the  modeling  of  joint  command  post  teams  is  still 
very  much  in  its  infancy  and  this  lack  of  foundational  research  hinders  our  ability  to  assess  the 
performance  of  command  post  teams1  and  to  assess  the  ability  of  a  new  technology  or 
procedures  to  improve  command  post  operations.  Therefore,  determining  how  to  model 
command  posts  is  an  important  need.  While  the  challenges  to  efficient  network  centric  warfare 
command  post  operations  can  be  addressed  by  experimentation  and  exploration,  command  post 
simulation  in  turn  poses  its  own  set  of  challenges.  In  the  main,  these  challenges  arise  from  the 
differences  between  real-world  command  post  settings  and  the  environments  that  are  available 
within  even  the  best  simulation  environments  and  from  the  differences  between  modeling  team 
performance  and  modeling  individual  performance.  These  challenges  have  not  been  addressed  in 
sufficient  detail  to  date. 

In  our  experience,  if  a  command  post  simulation  is  to  be  useful,  a  critical  characteristic 
that  must  to  be  preserved  is  the  ability  to  translate  experimental  results  into  real-world  actions, 
which  requires  proper  experimental  design.  Specifically,  the  lessons  learned  and  insights  gained 
in  the  simulation  environment  must  be  able  to  be  exploited  and  implemented  in  the  real-world  in 
order  to  improve  the  capability  of  the  teams  modeled  in  the  command  post  simulation.  The 
challenges  faced  in  accomplishing  this  translation  are  developing  a  simulation  environment  that 
has  sufficient  fidelity  in  the  proper  components  so  that  useful  real-world  conclusions  can  be 
drawn  and  in  identifying  the  scale  at  which  conclusions  should  be  made  so  that  they  can  translate 
into  real-world  actionable  recommendations. 

In  this  paper,  we  discuss  our  experience  in  command  post  team  simulation  and  the 
insights  we  have  gained  into  command  post  team  simulation.  Section  One  of  the  paper  discusses 
the  motivation  for  our  work  and  the  benefits  we  hope  to  derive  from  this  research.  Section  Two 
presents  relevant  background  research  and  its  relationship  to  our  current  effort.  Section  Three 
discusses  the  insights  gained  into  command  post  simulation  as  a  result  of  the  research  project  and 
the  results  we  have  obtained.  Section  Four  contains  a  summary  and  suggestions  for  future  work. 

1.  Introduction 

Highly  efficient  command  post  teams  are  essential  to  efficient  execution  of  network 
centric  warfare  operations.  Unfortunately,  the  modeling  of  teams,  let  alone  command  post 


A  team  consists  of  two  or  more  individuals  working  cooperatively  to  achieve  a  common  goal. 


teams,  is  still  very  much  in  its  infancy  and  this  lack  of  foundational  research  hinders  our  ability 
to  assess  the  perfonnance  of  command  post  teams2  and  to  assess  the  ability  of  a  new 
technologies  or  procedures  to  improve  command  post  operations.  Therefore,  determining  how  to 
model  command  posts  is  an  important  need  and  the  current  inability  hinders  efficient  network 
centric  operations.  While  addressing  the  challenges  to  efficient  network  centric  warfare 
command  post  operations  can  be  accomplished  by  experimentation  and  exploration,  command 
post  simulation  poses  its  own  set  of  challenges.  As  with  network  centric  command  posts, 
network  centric  command  post  simulation  technological  challenges  have  not  been  addressed 
sufficiently  to  date.  In  the  main,  command  post  simulation  technological  challenges  arise  from 
the  differences  between  real-world  command  post  settings  and  the  fidelity  of  the  command  post 
environments  that  are  available  within  even  the  best  simulation  environments  and  from  the 
differences  between  modeling  team  performance  and  modeling  individual  performance.  The 
simulation  shortfalls  arise  from  differences  in  technology  and  equipment,  differences  in  number 
and  expertise  of  personnel,  and  differences  in  control  and  communication  abilities  and 
technology.  The  team  modeling  shortfalls  are  important  because  command  posts  are  comprised 
of  both  teams  and  individuals,  resulting  in  team-to-team,  team-to-individual,  and  individual-to- 
individual  interactions  occurring  simultaneously  while  the  command  post  is  in  operation.  The 
team  modeling  ability  shortfalls  arise  from  our  lack  of  knowledge  about  team  modeling,  about 
the  complex  interactions  between  individuals  in  the  team,  about  the  reinforcing  and 
countervailing  skills  within  the  team  and  how  they  affect  team  performance,  and  about  the 
varying  competencies  among  team  members  in  the  command  post.  While  the  challenges  of 
command  post  simulation,  especially  for  network  centric  warfare,  are  fonnidable,  our  experience 
has  demonstrated  that  they  are  not  insuperable  if  the  experiments  and  explorations  are  properly 
designed  and  if  the  conclusions  drawn  are  appropriate  in  light  of  the  fidelity  of  the  command 
post  simulation  environment.  This  paper  reports  on  our  experience  in  command  post  team 
simulation  and  the  insights  we  have  developed  based  on  our  experience. 

If  a  command  post  simulation  is  to  be  useful  for  improving  command  post  operations  in 
the  real  world,  a  critical  quality  that  must  be  present  in  the  simulation  environment  is  the  ability 
to  transform  experimental  results  into  real-world  actions3,  which  requires  proper  experimental 
design.  Simply,  the  lessons  learned  and  insights  gained  in  the  simulation  environment  must  be 
able  to  be  exploited  and  implemented  in  the  real  world  in  order  to  improve  the  capability  of  the 
command  post  teams  that  were  modeled  in  the  command  post  simulation.  The  challenges  faced 
in  accomplishing  this  transformation  include  developing  a  simulation  environment  that  has 
sufficient  fidelity  in  the  proper  components  so  that  useful  real-world  conclusions  can  be  drawn 
and  in  identifying  the  scale  at  which  conclusions  should  be  made  so  that  they  can  translate  into 
real-world  actionable  recommendations.  An  experiment  for  command  post  simulation  must 
address  three  critical  and  ubiquitous  simulation  shortfalls  in  order  to  allow  its  results  to  be 
employed  in  the  real-world.  The  simulation  environment  shortfalls  typically  encountered  are 
inability  to  replicate:  1)  the  actual  command,  control,  and  communications  (C3)  environment  that 
exists  in  the  real-world,  2)  the  actual  size  and  composition  of  the  command  posts  that  are 
simulated,  and  3)  a  scenario  that  is  of  sufficient  length  and  intricacy  to  assess  the  command  post 
team  response  to:  real-world  stress,  human  task  overload,  individual  expertise,  and  complexity. 
These  constraints  limit  our  ability  to  draw  conclusions  about  individual  perfonnance 


“  A  team  consists  of  two  or  more  individuals  working  cooperatively  to  achieve  a  common  goal. 
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The  guidance  in  the  NATO  Code  of  Best  Practices  for  C2  Assessment  is  foundational  for  our  research. 


enhancements,  but  do  not  prohibit  assessment  of  factors  that  can  improve  command  post  team 
performance  fl‘59l 

In  addition  to  the  shortfalls,  research  performed  to  date  indicates  that  there  are  several 
reasons  for  the  difficulty  encountered  in  transferring  the  results  from  command  post  team 
simulations  to  their  real-world  counterparts  [1'591.  One  of  the  most  serious  impediments  to 
employing  the  results  of  the  experiment  in  the  real-world  are  the  disparity  in  the  number  of 
participants  in  the  experiment  versus  the  real-world.  Other  impediments  are  the  difference  in 
equipment,  individuals,  team  competencies,  individual  motivations,  and  differences  in  mental 
models  between  simulation  world  team  members  and  real-world  command  post  members. 
Because  of  these  differences,  there  is  no  experiment  that  can  be  devised  that  uses  the  available 
manpower  and  equipment  and  that  has  the  fidelity  and  scope  of  the  real-world  activity.  As  a 
result,  valid  conclusions  concerning  methods  for  improving  individual  performance  in  the  real- 
world  generally  are  not  possible  to  achieve  while  at  the  same  time  detennining  means  to  improve 
command  post  team  performance.  Nevertheless,  in  spite  of  the  constraints,  the  question  remains; 
How  can  we  achieve  more  timely  and  accurate  actions  and  decisions  and  improved  command 
post  perfonnance?  To  address  this  question,  two  subsidiary  and  fundamental  questions  are 
evident.  The  first  question  concerns  which  command  and  control  structure  and  process  results  in 
the  best  anticipatory  response  to  precursor  events.  The  second  question  concerns  which 
command  and  control  structure  and  process  best  deals  with  managing/controlling  the  response  to 
precursor  events  and  to  events  themselves.  Therefore,  the  command  post  simulation  experiment 
design  must  focus  on  improvements  in  team  behavior  and  performance  since  the  conclusions 
reached  about  a  team  within  the  bounds  of  the  experiment  environment  can  translate  to  changes 
to  be  made  to  actual  command  post  processes,  procedures,  and  functionality  in  the  real-world. 

So,  even  though  we  cannot  replicate  the  entire  milieu  faced  by  a  command  post  team  in 
the  real-world,  the  literature  is  clear  that  if  we  preserve  the  necessary  components  of  the  real- 
world  environment  then  we  can  draw  useful  conclusions  using  the  experimental  simulation  world 
that  are  actionable  in  the  real-world.  Because  of  this  fact,  the  one  fidelity  that  is  crucial  to  the 
experimental  simulation  world  in  order  to  project  experimental  results  back  into  the  real-world  is 
the  fidelity  of  the  mental  model  of  the  battlespace  held  by  each  person  in  the  command  post. 
However,  the  mental  model  must  be  scaled  in  space  and  time  to  correspond  to  that  which  is 
possible  in  the  real  world  given  the  differences  in  equipment  and  personnel.  An  additional  issue 
that  is  important  is  the  ability  to  control  team  workload  and  thereby  determine  if  changes  to  the 
environment  result  in  improved  team  decisions  and  actions. 

Within  the  realm  of  command  post  experiments,  there  are  four  aspects  of  each 
experiment  that  must  be  considered:  1)  Sensors,  2)  Intelligence  reports,  3) 
Network/communications  devices,  and  4)  Personnel  who  perceive  environment,  make  decisions, 
and  take  actions.  All  four  aspects  are  nearly  independent,  that  is,  an  improvement  in  sensors 
may  but  does  not  necessarily  lead  to  improved  decisions.  Likewise,  improved  network  and 
communications  devices  may  or  may  not  result  in  better  decision-making.  The  greatest  inter¬ 
dependence  appears  to  be  between  intelligence  and  decision-making.  This  dependence,  in  turn, 
results  in  two  cases,  the  first  case  is  one  in  which  we  attempt  to  determine  how  to  improve  the 
factors  (sensors,  human  perception,  cognition,  and  event  correlation)  that  affect  decision-making 
for  command  and  control  (C2).  The  second  case  is  one  in  which  we  are  attempting  to  improve 
the  decision-process,  which  is  the  ability  of  decision-makers  to  react  to  events  and  the  ability  of 
decision-makers  to  correlate  events  so  as  to  detect  pre-cursor  events  by  correlating  activities  that 
signal  an  impending  event.  However,  due  to  the  simulation  environment  shortfalls  mentioned 


above,  we  can  only  address  these  two  cases  at  the  team  level  of  activity.  The  first  case  revolves 
around  determining  how  well  the  command  and  control  structure  and  process  are  able  to 
correlate  activities  in  the  environment  that  are  signals  of  events.  The  second  case  revolves 
around  determining  how  well  the  command  and  control  structure  and  process  is  able  to  move 
information  between  its  constituent  components  that  need  the  infonnation  at  any  point  in  time. 
As  a  result,  command  post  team  simulation  experiments  should  do  the  following:  1)  concentrate 
on  these  two  cases,  2)  assess  the  flow,  quality,  and  quantity  of  information,  3)  attempt  to  be 
equivalent  to  real-world  command  post  team  expertise,  numbers,  and  taskload  in  its  key  aspects, 
and  4)  attempt  to  match  the  equipment  available  in  the  command  post  being  simulated.  Our 
research  focused  on  means  for  insuring  that  these  challenges  are  considered  and  accounted  for  in 
the  experiment  design. 

The  paper  discusses  our  experience  and  resulting  insights  into  in  command  post  team 
simulation.  The  remainder  of  the  paper  is  organized  as  follows.  The  next  section  contains  a 
discussion  of  relevant  background  research  and  its  relationship  to  our  current  effort.  Section 
Three  contains  a  discussion  of  our  research  project  and  the  results  of  our  efforts.  Section  Four 
contains  a  summary  and  suggestions  for  future  work. 

2,  Background 

There  are  two  major  factors  that  detennine  if  and  how  well  command  post  simulation 
experimental  results  can  be  translated  into  the  real-world.  The  first  factor  is  the  difference 
between  individual  and  team  decision-making.  The  second  factor  is  the  difference  between  the 
simulation  environment  and  the  real-world. 

2.1  Individual  versus  Team  Decision-Making 

There  are  many  differences  that  can  be  noted  in  the  manner  in  which  individuals  and 
teams  perform  decision-making,  as  summarized  in  Table  lfl  59l  For  example,  individual 
decision-making  may  have  a  much  more  narrower  focus  and  may  not  consider  all  relevant  data 
when  compared  to  team  decision-making.  Additionally,  team  decision-making  is  usually  driven 
by  a  desire  to  build  a  degree  of  consensus;  whereas,  individual  decision-makers  do  not  have  that 
obstacle.  In  addition,  apparently  it  is  more  difficult  for  a  team  than  an  individual  to  assemble  a 
consistent  mental  model  of  the  battlespace  but  teams  are  more  likely  to  have  a  broader 
appreciation  for  the  complexities  of  the  battlespace  than  an  individual’s  inherent  limitations 
allow  them  to  attain.  Experimentation  has  determined  that  individuals  are  limited  to  being  able 
to  accommodate  (attend  to)  seven  (plus  or  minus  two)  items  in  mind  at  any  instant  ^0|.  No 
equivalent  formulation  that  describes  a  team’s  attention  has  been  developed;  but  it  would  seem  to 
be  greater  due,  no  doubt,  to  the  ability  of  the  people  in  the  team  to  overlap  and  reinforce  items  to 
which  their  individual  concerns  are  addressed  (or  attended).  On  the  other  hand,  individuals  can 
readily  attend  to  one  item  and  exclude  all  other  matters  from  attention;  individuals  have  a  well- 
documented  ability  to  focus  their  attention  on  one  item  to  the  exclusion  of  all  others. 
Conversely,  it  is  apparently  very  difficult  for  an  entire  team  to  focus  its  attention  on  one  item 
(and  in  many  circumstances  it  is  counterproductive  to  do  so),  providing  both  the  promise  that 
important  activities  are  not  overlooked  but  also  threatening  to  dilute  the  effectiveness  of  the 
actions  and  decisions  of  the  team.  The  difference  in  ability  to  attend  to  one  matter  versus  several 
corresponds  to  the  difference  in  the  number  of  active  decision  processes  in  an  individual  and  a 
team  at  any  time.  Generally,  an  individual  can  pursue  one  active  decision  process,  whereas  a 
team  can  have  several.  Interestingly,  the  literature  suggests  that  both  individuals  and  teams  are 
subject  to  the  problem  of  process  interruption;  that  is,  when  a  process  is  underway  (whether  it  is 


a  decision  or  information  flow)  and  it  is  interrupted,  it  is  a  challenge  for  the  team  or  individual  to 
recover  from  the  interruption.  Though  it  appears  that  teams  may  be  able  to  recover  more  often 
than  individuals,  teams  are  also  subject  to  more  catastrophic  failures  when  interrupted  during  the 
execution  of  a  task  than  are  individuals  due  to  the  need  for  communication  and  a  shared 
understanding  of  the  state  of  task  accomplishment4. 

There  are  other  differences  in  the  characteristics  of  individuals  and  teams.  In  general,  an 
individual  has  one  skill  level  and  has  one  fatigue  level  at  a  given  time  and  they  change  relatively 
slowly.  As  a  result,  it  is  possible  to  characterize  a  person’s  response  to  events  and  potentially 
determine  how  to  improve  individual  perfonnance  when  fatigued,  under  stress,  or  when 
performance  does  not  meet  expectations.  On  the  other  hand,  a  team  has  multiple  skill  levels  in 
effect  concurrently  as  well  as  multiple  fatigue  factors  and  it  is  not  clear  how  to  combine  them  or 
ameliorate  their  effects  in  order  to  improve  team  performance.  Finally,  an  individual  remains 
relatively  “constant”  in  their  performance  over  the  duration  of  a  simulation  experiment,  a  team 
does  not  appear  to  remain  “constant”  since  the  team  can  vary  (in  composition  and  therefore 
“skill”  and  “perfonnance,”  as  well  as  in  group  dynamic  and  teamwork)  over  the  duration  of  a 
simulation  experiment. 


Characteristic 

Individual 

Team 

Scope  of  detail 

Limited,  difficult  to 
consider/attend  to  many  details 
simultaneously 

Broad,  many  details  can  be 
considered  simultaneously 

Breadth  of  attention 

Limited,  7  +/-  2 

Unknown,  but  seems  to  scale 
with  size  of  team 

Focus  of  attention 

Can  be  highly  focused 

Counter-productive  to  focus  on 
one  item 

Rapidity  of  decision-making 

Personality  dependant 

Team  competence  and 
composition  dependant 

Number  of  active  decision 
processes 

One  or  few 

Multiple,  no  generally 
accepted  model 

Fatigue 

One  level 

No  certain  model 

Decision-making  model 

OODA 

None 

Decision-process  interruption 

Problem 

Problem 

Table  1:  Individual  versus  Team  Characteristics  for  Information  Management  and  Decision- 

Making 


These  differences  between  individuals  and  teams  in  both  the  real-world  and  the 
simulation  environment  serve  to  limit  the  types  of  conclusions  that  we  can  draw  from  our 
command  post  team  experiments.  These  differences  are  not  insuperable  hurdles  that  prevent  the 
exploitation  of  the  results  of  the  experiment  within  the  real-world  command  post  environments 
that  are  being  modeled  because  they  can  be  overcome  by  careful  experimental  design. 

2.2  Simulation  and  Real-World  Differences 

To  insure  that  command  post  experiments  have  utility  in  the  real-world,  the  experiment’s 
objectives,  available  equipment,  number  of  available  players,  and  player  expertise  must  be 


One  of  the  more  memorable  examples  of  this  problem  is  the  August  16,  1987  crash  of  a  Northwest  Airlines  MD-80  on  takeoff  from 
Detroit  International  Airport.  An  untimely  interruption  of  the  crew  during  execution  of  the  takeoff  checklist  resulted  in  each  crew  member 
thinking  that  someone  else  had  insured  that  the  wingflaps  were  down,  when  in  fact  they  had  never  been  extended. 


assessed  in  order  to  determine  how  they  would  affect  the  experiment  and  the  translation  of  its 
results  into  the  real-world.  Properly  addressing  these  issues  insures  that  the  experiment  has 
applicability  in  the  real-world;  ie.,  that  the  experiment  results  can  be  translated  from  the 
experimental  setup  into  real-world  operation.  These  differences  are  not  trivial  and  if  not 
properly  managed,  would  degrade  the  utility  and  applicability  of  the  experiment.  Current 
research  indicates  that  the  most  serious  impediments  to  employing  the  results  of  a  team 
simulation  experiment  in  the  real-world  are  the  disparity  in  the  number  of  participants  in  the 
experiment  versus  the  real-world  and  the  disparity  between  available  equipment  in  the 
experiment  versus  the  real-world.  Because  of  the  differences  in  available  manpower  and 
equipment,  it  is  not  possible  to  conduct  a  command  post  team  experiment  that  matches  both  the 
scope  and  the  complexity  of  the  corresponding  real-world  activity.  Because  we  are  interested  in 
improving  command  post  team  performance,  the  scope  of  activity  must  be  maintained  and 
individuals  must  perfonn  additional  tasks  within  the  team  than  they  would  be  called  upon  to 
perform  in  real-world  operations.  As  a  result,  valid  conclusions  concerning  methods  for 
improving  individual  perfonnance  in  the  real-world  can  not  be  drawn  within  a  command  post 
experiment;  however,  we  can  draw  valid  conclusions  concerning  methods  for  improving  team5 
performance  (where  a  team  is  a  command  post  entity).  Our  goal,  then,  for  developing  a 
command  post  team  experiment  and  for  insuring  that  experiment  results  can  translate  into  the 
real  world  is  to  insure  that  the  experiment  is  designed  so  that  essential  characteristics  of  the  team 
aspect  in  the  real-world  are  present  in  the  experiment.  Achieving  this  quality  of  simulation  is  a 
challenge  because  of  the  differences  between  the  simulation  environment  and  the  real-world. 

In  sum,  as  a  result  of  the  constraints  and  goals  identified  for  a  command  post  simulation 
experiment,  the  experiment  must  focus  on  improvements  in  team  behavior  and  performance  in 
support  of  decision-making.  This  focus  is  necessary  because  the  conclusions  reached  about  a 
team  within  the  bounds  of  the  experiment  environment  will  translate  to  changes  that  can  be  made 
to  actual  command  post  processes,  procedures,  and  functionality  in  the  real-world.  Because  of 
the  differences  between  the  simulation  environment  and  the  real-world,  improvements  in 
individual  performance  in  the  simulation  environment  should  not  be  assumed  to  result  in 
improved  individual  or  team  performance  in  the  real-world. 

3,  Discussion 

To  assess  the  performance  of  a  command  post  team,  two  types  of  information  are 
essential.  The  first  type  of  information  is  an  understanding  of  the  factors  that  affect  command 
post  team  performance  and  the  second  type  of  information  is  a  set  of  metrics  for  assessing  the 
performance. 


3.1  Team  Performance  Considerations 

Because  the  command  post  team  level  is  the  domain  of  interest  for  the  experimental 
results,  the  guiding  imperative  was  to  insure  that  the  simulation  command  post  team  and  the  real- 
world  command  post  team  were  as  similar  as  possible  in  all  of  the  key  areas  that  affect  team 
performance.  Within  the  design  of  command  post  experiments,  as  we  have  noted  above,  there 
are  four  factors  that  affect  command  post  team  perfonnance:  1)  sensors,  2)  intelligence  reports, 


We  use  the  term  team  instead  of  group  because  of  the  differences  in  connotation  that  these  words  have  in  the  research 
community  and  everyday  usage.  A  team  is  generally  considered  to  be  composed  of  a  number  of  individuals,  working  toward  a 
common  purpose,  each  with  assigned  tasks,  under  some  form  of  leadership/command/authority.  A  group,  on  the  other  hand,  is 
unstructured,  has  a  vaguely  defined  purpose,  shifting  responsibilities,  and  no  clear  authority. 


3)  network/communications  devices,  and  4)  personnel  who  make  decisions,  and  take  actions.  All 
four  factors  are  nearly  independent.  For  example,  an  improvement  in  sensors  may  but  does  not 
necessarily  correlate  with  improved  decisions  and  actions  by  personnel.  Likewise,  improved 
network  and  communications  devices  may  but  do  not  necessarily  correlate  with  improved 
personnel  decision-making.  However,  there  is  an  inter-dependence  between  intelligence 
reporting  and  personnel  decision-making  in  command  post  team  performance. 

3.1.1  Team  Performance  Areas  to  be  Evaluated 

The  research  literature  and  analysis  indicates  that  there  are  two  broad  areas  of  command 
post  team  evaluation  that  must  be  examined  experimentally.  The  first  experimental  area  is  the 
investigation  of  the  four  factors  separately.  The  second  experimental  area  is  examination  of  the 
interconnection  between  intelligence  and  personnel  decisions  and  actions.  Fortunately,  the  two 
areas  are  independent  enough  so  that  they  can  be  examined  simultaneously;  ie.,  one  experiment 
can  test  hypotheses  for  both  areas  simultaneously  without  corrupting  the  experiment  results  for 
either  area. 

When  attempting  to  gain  insight  into  the  first  area,  experiments  must  be  performed  to 
determine  how  to  improve  each  of  the  four  factors  that  affect  command  post  team  perfonnance. 
When  attempting  to  gain  insight  into  the  second  area,  experiments  must  be  performed  to 
detennine  how  to  improve  the  processes  for  gathering,  handling,  examining,  and  managing 
intelligence  that  affect  command  post  team  performance.  In  sum,  command  post  team 
simulation  experiments  should  do  the  following:  1)  concentrate  on  these  two  cases,  2)  assess  the 
flow,  quality,  and  quantity  of  infonnation,  3)  attempt  to  match  real-world  command  post  team 
expertise,  numbers,  and  taskload,  especially  in  the  areas  of  command  post  performance  that  are 
central  to  the  experiment,  and  4)  attempt  to  match  the  equipment  available  in  the  command  post 
being  simulated.  For  both  cases,  quality  of  intelligence  reporting  may  be  a  factor  and  should  be 
explicitly  considered  in  the  experimental  design. 

3.1.2  Assess  Impact  of  Intelligence  Upon  Team  Performance 

The  research  literature  and  analysis  indicated  that  within  the  area  of  command  post  team 
operations  dealing  with  intelligence  that  there  are  two  distinct  cases  of  intelligence  input  to 
consider.  The  first  intelligence  case  revolves  around  detennining  how  well  the  command  and 
control  structure  and  process  are  able  to  correlate  activities  in  the  environment  that  are  precursors 
to  events.  A  mix  of  intelligence  inputs  are  needed  because  the  experiment  is  actually  trying  to 
determine  how  well  a  significant  event  can  be  separated  from  background  infonnation  (aka 
noise).  False/inac curate  intelligence  would  be  a  normal  part  of  that  background,  so  the  ability  to 
detect  and  reject  false  positives  as  well  as  insure  that  detection  of  true  precursors  is  detennined. 
The  measures  of  command  and  control  (C2)performance  that  must  be  applied  in  this  case  are 
those  that  assess  the  command  and  control  (C2)  structure’s  ability  to  determine  the 
accuracy/validity  of  the  information  that  it  has  and  how  well  a  given  C2  structure  and  processes 
supports  the  information  determinations  that  must  be  made.  The  second  intelligence  case 
revolves  around  determining  how  well  the  command  and  control  structure  and  process  is  able  to 
move  infonnation  between  the  command  post  components  that  need  the  information  at  any  point 
in  time.  In  this  experiment  case,  the  goal  is  measuring  the  latency  between  the  time  when 
information  becomes  available  until  all  portions  of  the  command  post  structure  that  need  the 
information  have  the  information. 

3.2  Team  Performance  Metrics 

When  considering  experimental  area  one,  the  metrics  to  be  used  vary  depending  upon  the 
intelligence  case  being  considered.  For  intelligence  case  one  in  experimental  area  one,  potential 


metrics  include  but  are  not  limited  to  the  following:  1)  elapsed  time,  2)  number  of  participants  in 
the  process,  3)  number  of  extraneous  participants  in  the  process,  4)  number  of  precursor  events 
that  are  addressed  by  the  C2  structure,  5)  number  of  alternative  explanations  for  a  precursor 
event  that  are  considered,  6)  criteria  applied  to  determine  the  significance  of  a  precursor  event,  7) 
the  number  of  times  that  the  explanation  assigned  to  a  precursor  event  by  a  given  C2  structure  is 
accurate,  8)  the  “difficulty”  faced  when  detennining  what  a  precursor  event  signifies,  9)  the 
number  of  times  a  sensor  is  re-tasked  correctly,  10)  the  number  of  times  a  sensor  is  re-tasked 
incorrectly  (unnecessarily),  11)  the  “quality”  of  the  decision/interpretation,  and  12)  the  overall 
“quality”  of  decision  process.  For  intelligence  case  two  in  experimental  area  one,  potential 
metrics  include  the  following:  1)  the  speed  of  the  movement  of  information  through  the 
command  post  and  C2  structure,  2)  the  number  of  people  who  should  have  the  information  but 
did  not  get  it  (directly),  3)  the  number  of  people  who  did  not  need  the  information  but  received  it, 
4)  the  information  volume,  5)  the  “quality”  of  communication,  6)  the  number  of  times  that 
information  has  to  be  retransmitted,  and  7)  the  latency  between  when  a  re-tasking  directive  is 
sent  and  when  the  re-tasking  is  accomplished. 

When  considering  experimental  area  two,  the  metrics  to  be  used  do  not  vary  depending 
upon  the  intelligence  case  being  considered.  For  this  experiment  area,  potential  metrics  include 
but  are  not  limited  to  the  following:  1)  the  elapsed  time  to  make  a  decision,  2)  the  “quality”  of  a 
decision,  3)  the  number  of  alternative  responses  considered,  4)  the  “quality”  of  processes 
employed,  5)  the  number  of  people  involved  in  the  decision  process,  6)  the  number  of  extraneous 
participants  in  the  process,  7)  the  criteria  applied  to  determine  a  response,  8)  the  “difficulty”  of 
determining  the  correct  response,  9)  the  number  of  pieces  of  infonnation  considered  when 
formulating  a  response,  10)  the  number  of  pieces  of  extraneous  infonnation  considered  when 
formulating  a  response,  and  11)  the  overall  assessment  of  the  command  post’s  “understanding” 
of  the  battlespace. 

Clearly,  assessing  command  post  team  perfonnance  is  a  challenge  and  requires  careful 
planning  for  the  experiment  (The  NATO  Code  of  Best  Practices  for  C2  Assessment  is  a  useful 
aid),  determination  of  the  data  to  be  collected,  and  development  of  a  data  analysis  plan.  In 
addition,  to  improve  the  validity  of  the  command  post  simulation  experiment  it  should  be 
repeated  as  much  as  practicable  in  order  to  improve  the  statistical  validity  of  the  results. 

4.  Summary  and  Future  Work 

Highly  efficient  command  post  teams  are  essential  to  efficient  execution  of  military 
operations.  Unfortunately,  the  modeling  of  joint  command  post  teams  is  still  a  new  field  and  the 
lack  of  foundational  research  hinders  our  ability  to  assess  the  perfonnance  of  command  post 
teams  and  to  assess  the  ability  of  a  new  technology  or  procedure  to  improve  command  post 
operations.  Therefore,  determining  how  to  model  command  posts  is  an  important  research 
question.  To  enable  the  translation  of  the  simulation  environment’s  experimental  results  into  the 
real-world,  the  simulation  command  post  team  the  infonnation  flow  must  be  essentially  identical 
to  the  information  flow  in  the  real-world  command  post. 

In  this  paper,  we  presented  our  experience  in  command  post  team  simulation  and  the 
insights  gained  into  command  post  team  simulation.  We  briefly  examined  relevant  background 
research  and  its  relationship  to  command  post  simulation.  We  also  discussed  insights  gained  as  a 
result  of  our  research  project  and  results.  However,  many  research  questions  related  to  command 
post  simulation  and  effective  command  post  operation  remain  to  be  addressed.  One  of  the 
challenges  arises  from  the  need  to  learn  how  to  exploit  the  coming  capabilities  for  improved 


communication.  These  improved  communication  capabilities  can  be  used  to  improve  command 
post  operation  and  command  and  control  structures  as  well  as  to  improve  understanding  about 
the  battlespace.  Another  challenge  originates  from  the  need  to  improve  the  structure  and 
composition  of  the  command  post  teams  so  that  the  various  teams  that  comprise  a  command  post 
increase  their  efficiency,  both  individually  and  in  the  aggregate.  Other  challenges  arise  from  the 
need  to  leam  how  to  increase  command  post  operational  efficiency  and  how  to  conduct 
command  post  operations  in  a  network  centric  environment.  These  achievements  must  also  be 
coupled  with  the  drive  to  reduce  the  number  of  people  who  are  used  to  operate  a  command  post. 
These  and  other  challenges  to  command  post  team  operations  can  be  addressed  by  using 
simulation  experiments  to  evaluate  alternative  command  post  configurations. 
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